Reproducing Kernel Hilbert Space Approach Research Articles

Semiparametric estimation for the functional additive hazards model

AbstractWe propose a new functional additive hazards model to investigate the potential effects of functional and scalar predictors on mortality risks, and develop a penalized least squares estimation method for model parameters based on a pseudoscore estimating equation. A reproducing kernel Hilbert space approach is used to establish the consistency, convergence rate, and joint asymptotic distribution of the resulting estimators for finite‐dimensional and infinite‐dimensional parameters. Our simulation studies demonstrate that the proposed estimation procedure performs well. For illustration, we apply the proposed method to the Medical Information Mart for Intensive Care III dataset.

Functional Varying Coefficient Model with Time-independent Covariate and Longitudinal Response

In this paper, we consider functional varying coefficient model in present of a time invariant covariate for sparse longitudinal data contaminated with some measurement errors. We propose a regularization method to estimate the slope function based on a reproducing kernel Hilbert space approach. As we will see, our procedure is easy to implement. Our simulation results show that the procedure performs well, especially when either sampling frequency or sample size increases. Applications of our method are illustrated in an analysis of a longitudinal CD4+ count dataset from an HIV study.

Robust multivariate nonparametric tests via projection averaging

In this work, we generalize the Cramér–von Mises statistic via projection averaging to obtain a robust test for the multivariate two-sample problem. The proposed test is consistent against all fixed alternatives, robust to heavy-tailed data and minimax rate optimal against a certain class of alternatives. Our test statistic is completely free of tuning parameters and is computationally efficient even in high dimensions. When the dimension tends to infinity, the proposed test is shown to have comparable power to the existing high-dimensional mean tests under certain location models. As a by-product of our approach, we introduce a new metric called the angular distance which can be thought of as a robust alternative to the Euclidean distance. Using the angular distance, we connect the proposed method to the reproducing kernel Hilbert space approach. In addition to the Cramér–von Mises statistic, we demonstrate that the projection-averaging technique can be used to define robust multivariate tests in many other problems.

A reproducing kernel Hilbert space approach to high dimensional partially varying coefficient model

Partially varying coefficient model (PVCM) provides a useful class of tools for modeling complex data by incorporating a combination of constant and time-varying covariate effects. One natural question is that how to decide which covariates correspond to constant coefficients and which correspond to time-dependent coefficient functions. To handle this two-type structure selection problem on PVCM, those existing methods are either based on a finite truncation way of coefficient functions, or based on a two-phase procedure to estimate the constant and function parts separately. This paper attempts to provide a complete theoretical characterization for estimation and structure selection issues of PVCM, via proposing two new penalized methods for PVCM within a reproducing kernel Hilbert space (RKHS). The proposed strategy is partially motivated by the so-called “Non-Constant Theorem” of radial kernels, which ensures a unique and unified representation of each candidate component in the hypothesis space. Within a high-dimensional framework, minimax convergence rates for the prediction risk of the first method is established when each unknown time-dependent coefficient can be well approximated within a specified RKHS. On the other hand, under certain regularity conditions, it is shown that the second proposed estimator is able to identify the underlying structure correctly with high probability. Several simulated experiments are implemented to examine the finite sample performance of the proposed methods.

Supervised domain generalization for integration of disparate scalp EEG datasets for automatic epileptic seizure detection

Numerous automatic epileptic seizure detectors (ESDs) with excellent performances have been reported, but they generally experience performance degradation when tested with real-life clinical data. This has been blamed on the scarcity of high-quality training data, which leads to models that generalize poorly. There is consequently interest in methods to improve the quality and quantity of training data for ESDs. This study used a domain generalization approach to combine data from two different datasets for training an ESD, which was thereafter tested on a third dataset. A subspace of the CHB-MIT and TUSZ scalp EEG seizure datasets was extracted using transfer component analysis, based on a reproducing kernel Hilbert space approach. We then used the Azimuthal Equidistant Projection to transform 3D electrode coordinates into 2D space, followed by interpolation using the Clough–Tocher technique to generate 16x16 rasters. We thereafter generated feature vectors, each of which was a sequence of 17 ten-layer 16x16 raster arrays. The vectors were used to train a recurrent-convolutional neural network. The network had a 128-unit long short-term memory layer with inputs from 17 parallel networks each with three stacks of convolutional layers. Testing was based on a private 26-subject dataset, combined with randomly selected subsets of the CHB-MIT and TUSZ datasets. A combined sensitivity of 74.5% was achieved, along with a false positive per hour rate of 0.84, and a latency of 2.32 s. Detection sensitivity on the private dataset was 72.5%. These results compare favorably with results of large-scale validation studies in literature and confirm the viability of this approach to increasing the size of training datasets for ESDs.

Reactive Navigation Under Non-Parametric Uncertainty Through Hilbert Space Embedding of Probabilistic Velocity Obstacles

The probabilistic velocity obstacle (PVO) extends the concept of velocity obstacle (VO) to work in uncertain dynamic environments. In this paper, we show how a robust model predictive control (MPC) with PVO constraints under non-parametric uncertainty can be made computationally tractable. At the core of our formulation is a novel yet simple interpretation of our robust MPC as a problem of matching the distribution of PVO with a certain desired distribution. To this end, we propose two methods. Our first baseline method is based on approximating the distribution of PVO with a Gaussian Mixture Model (GMM) and subsequently performing distribution matching using Kullback Leibler (KL) divergence metric. Our second formulation is based on the possibility of representing arbitrary distributions as functions in Reproducing Kernel Hilbert Space (RKHS). We use this foundation to interpret our robust MPC as a problem of minimizing the distance between the desired distribution and the distribution of the PVO in the RKHS. Both the RKHS and GMM based formulation can work with any uncertainty distribution and thus allowing us to relax the prevalent Gaussian assumption in the existing works. We validate our formulation by taking an example of 2D navigation of quadrotors with a realistic noise model for perception and ego-motion uncertainty. In particular, we present a systematic comparison between the GMM and the RKHS approach and show that while both approaches can produce safe trajectories, the former is highly conservative and leads to poor tracking and control costs. Furthermore, RKHS based approach gives better computational times that are up to one order of magnitude lesser than the computation time of the GMM based approach.

The reproducing kernel Hilbert space approach in nonparametric regression problems with correlated observations

In this paper, we investigate the problem of estimating the regression function in models with correlated observations. The data are obtained from several experimental units, each of them forms a time series. Using the properties of the reproducing kernel Hilbert spaces, we construct a new estimator based on the inverse of the autocovariance matrix of the observations. We give the asymptotic expressions of its bias and its variance. In addition, we give a theoretical comparison between this new estimator and the popular one proposed by Gasser and Müller, we show that the proposed estimator has an asymptotically smaller variance than the classical one. Finally, we conduct a simulation study to investigate the performance and the robustness of the proposed estimator and to compare it to the Gasser and Müller’s estimator in a finite sample set.

On estimation in varying coefficient models for sparse and irregularly sampled functional data

In this paper, we study a smoothness regularization method for a varying coefficient model based on sparse and irregularly sampled functional data which is contaminated with some measurement errors. We estimate the one-dimensional covariance and cross-covariance functions of the underlying stochastic processes based on a reproducing kernel Hilbert space approach. We then obtain least squares estimates of the coefficient functions. Simulation studies demonstrate that the proposed method has good performance. We illustrate our method by an analysis of longitudinal primary biliary liver cirrhosis data.

A reproducing kernel Hilbert space approach in meshless collocation method

In this paper, we combine the theory of the reproducing kernel Hilbert spaces with the field of collocation methods to solve boundary value problems with a special emphasis on the reproducing property of kernels. Using the reproducing property of the kernels, a new efficient algorithm is proposed to obtain the cardinal functions of a reproducing kernel Hilbert space, which can be applied conveniently for multi-dimensional domains. The differentiation matrices are constructed and also a pointwise error estimate of applying them is derived. In addition, we prove the non-singularity of the collocation matrix. The proposed method is truly meshless, and can be applied conveniently and accurately for high order and also multi-dimensional problems. Numerical results are presented for the several problems such as second- and fifth-order two-point boundary value problems, one- and two-dimensional unsteady Burgers’ equations, and a three-dimensional parabolic partial differential equation. In addition, we compare the numerical results with the best-reported results in the literature to show the high accuracy and efficiency of the proposed method.

An RKHS approach to estimate individualized treatment rules based on functional predictors

In recent years there has been massive interest in precision medicine, which aims to tailor treatment plans to the individual characteristics of each patient. This paper studies the estimation of individualized treatment rules (ITR) based on functional predictors such as images or spectra. We consider a reproducing kernel Hilbert space (RKHS) approach to learn the optimal ITR which maximizes the expected clinical outcome. The algorithm can be conveniently implemented although it involves infinite-dimensional functional data. We provide convergence rate for prediction under mild conditions, which is jointly determined by both the covariance kernel and the reproducing kernel.

Simultaneous estimations of optimal directions and optimal transformations for functional data

Sliced inverse regression (SIR) is a powerful method to deal with a dimension reduction model. As well known, SIR is equivalent to a transformation-based projection pursuit, where the optimal directions are just the directions in SIR. In this paper, we consider simultaneous estimations of optimal directions for functional data and optimal transformations. We take a reproducing kernel Hilbert space approach. Both the directions and the transformations are chosen from reproducing kernel Hilbert spaces. A learning rate is established for the estimators.

The C(3P) + NO(X2Π) O(3P) + CN(X2Σ+), N(2D)/N(4S) + CO(X1Σ+) reaction: Rates, branching ratios, and final states from 15 K to 20 000 K.

The C + NO collision system is of interest in the area of high-temperature combustion and atmospheric chemistry. In this work, full dimensional potential energy surfaces for the 2A', 2A″, and 4A″ electronic states of the [CNO] system have been constructed following a reproducing kernel Hilbert space approach. For this purpose, more than 50 000 ab initio energies are calculated at the MRCI+Q/aug-cc-pVTZ level of theory. The dynamical simulations for the C(3P) + NO(X2Π) → O(3P) + CN(X2Σ+), N(2D)/N(4S) + CO(X1Σ+) reactive collisions are carried out on the newly generated surfaces using the quasiclassical trajectory (QCT) calculation method to obtain reaction probabilities, rate coefficients, and the distribution of product states. Preliminary quantum calculations are also carried out on the surfaces to obtain the reaction probabilities and compared with QCT results. The effect of nonadiabatic transitions on the dynamics for this title reaction is explored within the Landau-Zener framework. QCT simulations have been performed to simulate molecular beam experiment for the title reaction at 0.06 and 0.23 eV of relative collision energies. Results obtained from theoretical calculations are in good agreement with the available experimental as well as theoretical data reported in the literature. Finally, the reaction is studied at temperatures that are not practically achievable in the laboratory environment to provide insight into the reaction dynamics at temperatures relevant to hypersonic flight.

A unified penalized method for sparse additive quantile models: an RKHS approach

This paper focuses on the high-dimensional additive quantile model, allowing for both dimension and sparsity to increase with sample size. We propose a new sparsity-smoothness penalty over a reproducing kernel Hilbert space (RKHS), which includes linear function and spline-based nonlinear function as special cases. The combination of sparsity and smoothness is crucial for the asymptotic theory as well as the computational efficiency. Oracle inequalities on excess risk of the proposed method are established under weaker conditions than most existing results. Furthermore, we develop a majorize-minimization forward splitting iterative algorithm (MMFIA) for efficient computation and investigate its numerical convergence properties. Numerical experiments are conducted on the simulated and real data examples, which support the effectiveness of the proposed method.

RKHS approach for signal detection in rotation and scale space random fields

Two important papers of Worsley, Siegmund and coworkers consider rotation and scale space random fields for detecting signals in fMRI (functional magnetic resonance imaging) brain images. They use the global maxima of images for detection of a signal. In the current work, we utilize a reproducing kernel Hilbert space (RKHS) approach to show for both rotation and scale space random fields the global maximum of the image is indeed the likelihood ratio test statistic.

A reproducing kernel Hilbert space constructive approximation for a special class of integral equations

A reproducing kernel Hilbert space approach is proposed to study a class of integral equations with Toeplitz and Hankel kernel functions. The existence property and approximate representations of the solutions are given by constructing appropriate auxiliary operators and positive definite matrices within a reproducing kernel Hilbert space framework. Moreover, conditions for the boundedness and uniqueness of the solution are also obtained.

The RKHS Approach to Minimum Variance Estimation Revisited: Variance Bounds, Sufficient Statistics, and Exponential Families

The mathematical theory of reproducing kernel Hilbert spaces (RKHSs) provides powerful tools for minimum variance estimation (MVE) problems. Here, we extend the classical RKHS-based analysis of MVE in several directions. We develop a geometric formulation of five known lower bounds on the estimator variance (Barankin bound, Cramér-Rao bound, constrained Cramér-Rao bound, Bhattacharyya bound, and Hammersley-Chapman-Robbins bound) in terms of orthogonal projections onto a subspace of the RKHS associated with a given MVE problem. We show that, under mild conditions, the Barankin bound (the tightest possible lower bound on the estimator variance) is a lower semicontinuous function of the parameter vector. We also show that the RKHS associated with an MVE problem remains unchanged if the observation is replaced by a sufficient statistic. Finally, for MVE problems conforming to an exponential family of distributions, we derive novel closed-form lower bounds on the estimator variance and show that a reduction of the parameter set leaves the minimum achievable variance unchanged.

Supervised and transductive multi-class segmentation using p-Laplacians and RKHS methods

This paper considers supervised multi-class image segmentation: from a labeled set of pixels in one image, we learn the segmentation and apply it to the rest of the image or to other similar images. We study approaches with p-Laplacians, Reproducing Kernel Hilbert Spaces (RKHSs) and combinations of both. In all approaches we construct segment membership vectors. In the p-Laplacian model the segment membership vectors have to fulfill a certain probability simplex constraint. Interestingly, we could prove that this is not really a constraint in the case p=2 but is automatically fulfilled. While the 2-Laplacian model gives a good general segmentation, the case of the 1-Laplacian tends to neglect smaller segments. The RKHS approach has the benefit of fast computation. We further consider an improvement by combining p-Laplacian and RKHS methods. Finally, we present challenging applications to medical image segmentation.

Reproducing kernel Hilbert space based estimation of systems of ordinary differential equations

Non-linear systems of differential equations have attracted the interest in fields like system biology, ecology or biochemistry, due to their flexibility and their ability to describe dynamical systems. Despite the importance of such models in many branches of science they have not been the focus of systematic statistical analysis until recently. In this work we propose a general approach to estimate the parameters of systems of differential equations measured with noise. Our methodology is based on the maximization of the penalized likelihood where the system of differential equations is used as a penalty. To do so, we use a reproducing kernel Hilbert space approach that allows us to formulate the estimation problem as an unconstrained numeric maximization problem easy to solve. The proposed method is tested with synthetically simulated data and it is used to estimate the unobserved transcription factor cdaR in Streptomyces coelicolor using gene expression data of the genes it regulates.

Discrete linear differential equations

We propose new constructions of the approximate solutions of discrete linear differential equations and their inverse source problems. This is mainly based on a reproducing kernel Hilbert spaces approach where different types of spaces are naturally considered as a consequence of the method. Here, the major influence will be given by the Paley–Wiener and Sobolev spaces.

A simple NSC-RKHS algorithm for solving linear boundary value problems – Error bounds and implementations

The simple algorithm is in essence a normal spline collocation method introduced by V.K. Gorbunov in 1989. The paper presents a reproducing kernel Hilbert space approach to the method, that gives rise to basic error inequalities. For a natural H2-class of mth order linear problems the approximation of any normal solution v is Cm-1-uniform, i.e. v(a)(t)-vn(a)(t)⩽caΔn2‖v‖Hm+2, for a<m, where Δn is diameter of the partition of the domain by nodes. Simplicity of the algorithm in Sobolev spaces is justified by explicit formulas for the kernel functions – discovered by Gorbunov and V.V. Petrishchev in 2003 and underestimated in the opinion of the author. As an application, numerical solutions in H5 and in H6 are compared with those computed by other authors. The simple method is then applied to a fifth-order nonlocal boundary value problem – in H6 and in H7.